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Abstract 


A  new  technique  for  Identifying  inefficient  hospitals,  Data  Envelopment 
Analysis  (DEA) ,  is  field  tested  by  application  to  a  group  of  teaching 
hospitals.  DEA  is  found  to  provide  meaningful  insights  into  the  location  and 
nature  of  hospital  inefficiencies  as  judged  by  the  opinion  of  a  panel  of 
hospital  experts.  The  insights  about  hospital  efficiency  provided  by  DEA  are 
not  generated  from  the  widely  used  efficiency  evaluation  techniques  of  ratio 
analysis  and  econometric-regression  analysis.  DEA  is,  therefore,  suggested  as 
a  means  to  help  identify  and  measure  hospital  inefficiency  as  a  basis  for 
directing  management  efforts  toward  increasing  efficiency  and  reducing  health 
care  costs. 
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1,  Introduction 

One  approach  that  a  hospital  administrator  or  regulator  can  pursue  to 
reduce  health  care  costs  is  to  reduce  inefficiencies  In  hospital  operations. 
Only  after  hospital  inefficiencies  are  identified  and  measured  can  remedial 
action  be  taken  to  Improve  efficiency  and  thereby  reduce  hospital  operating 
costs*  This  paper  field  tests  a  new  approach  to  Identify  and  measure  hopsital 
inefficiencies,  Data  Envelopment  Analysis  (DEA) ,  which  provides  insights  that 
are  not  available  from  currently  used  techniques  and  which  can  form  the  basis 
for  managerial  action  to  Improve  hospital  operating  efficiency. 

The  following  section  describes  the  need  for  applying  such  efficiency 
evaluation  techniques  to  hospitals.  Section  3  describes  the  weaknesses  in 
currently  used  techniques  for  measuring  hopltal  efficiency  and  the  ways  in 
which  DEA  circumvents  these  weaknesses.  Section  4  describes  the  application 
of  DEA  to  a  set  of  teaching  hospitals  and  the  evaluation  of  the  results  by  a 
panel  of  hospital  experts  and  by  management  of  one  of  the  hospitals  identified 
as  Inefficient.  The  final  section  5  discusses  the  strengths  and  limitations 
of  DEA  and  the  ways  it  can  be  useful  in  future  hospital  application. 

2,  Efficiency  Measurement  Techniques  are  needed  to  help  reduce  health  care 
costs 

Rising  health  care  costs  are  of  widespread  concern  to  government, 
individuals,  and  insurers.  In  the  past,  hospitals  generality  were  reimbursed 
for  services  based  on  the  cost  of  providing  those  services.  This  type  of 
retrospective  reimbursement  does  not  provide  strong  incentives  to  reduce 
hospital  costs,  i.e.,  it  allows  hospitals  to  recover  their  costs  regardless  of 


their  level  of  efficiency.  Recently,  reimbursement  systems  have  been 
attempting  to  reduce  the  rate  of  increase  in  hospital  costs  through 
prospective  rate  setting  and  reimbursement  for  hospital  services  [A]  [15]. 
Propective  systems  basically  require  a  hospital  to  agree  on  a  reimbursement 
rate  structure  for  services  to  be  rendered  in  the  future  based  on  budgets 
prepared  by  a  hospital  and  accepted  by  the  regulator  and/or  payor.  While 
several  studies  (e.g.,  [4]  and  [15])  suggest  that  prospective  reimbursement 
systems  lead  to  lower  rates  of  cost  increases,  they  do  not  create  the  type  of 
direct  price  competition  which  is  expected  to  induce  efficient  behavior  (in 
the  microeconomic  theory  sense).  Hospitals  do  compete  on  many  dimensions  for 
financing,  donations,  patients,  and  affiliations  with  medical  schools  and 
physicians,  but  success  along  these  dimensions  of  competition  would  not 
necessarily  result  in  efficient  hospital  operations.  While  the  new 
prospective  reimbursement  mechanisms  provide  incentives  to  prevent  a  hospital 
from  becoming  less  efficient,  they  do  not  assure  that  hospitals  will  become 
efficient  partly  because  strategies  to  maximize  reimbursement  may  not  lead  to 
improvement  of  efficiency  and  partly  because  prospective  reimbursement  systems 
are  largely  based  on  current  cost  levels  which  are  not  necessarily  efficient 
cost  levels.  That  is,  prospective  systems  may  only  motivate  a  hospital  to 
limit  the  rate  of  cost  increases  from  an  already  inefficient  cost  level. 
Hence  other  more  direct  approaches  to  improve  hospital  efficiency  are  needed 
as  one  means  of  reducing  health  care  costs. 

Can  hospitals  reduce  operating  costs  by  improving  operating  effciency? 
The  answer  to  this  question  is,  at  worst,  unclear  and,  at  best,  yes.  The 
efficient  amount  of  resources  required  to  produce  hospital  services  is  not 
well  understood  or  specifiable  in  any  detail  ([2],  [15]  and  [9]).  If  the 
efficient  amount  of  resources  needed  to  produce  each  type  of  patient  care  were 
known,  the  efficient  cost  of  providing  hospital  services  could  be  directly 
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determined.  Numerous  studies  (e.g.,  [9])  have  attempted  to  identify  the 
efficient  hospital  production  relationships  using  econometric-regression 
techniques.  These  studies  share  a  common  methodological  weakness  in  that  they 
are  based  on  estimating  average  input-output  relationships  based  on  data  which 
includes  both  efficient  and  inefficient  hospitals.  They  may  provide  good 
predictions  of  what  costs  will  be  assuming  a  constant  level  of  inefficiency 
but  they  say  nothing  about  efficient  relationships  ([14]). 

Other  studies  suggests  that  many  hospitals  are  operating  with  varying 
degrees  of  inefficiency  which  could  be  reduced  by  managerial  action  if  the 
proper  incentives  existed.  The  U.S.  General  Accounting  Office  (GAO)  study  of 
U.S.  hospitals  [15]  Indicated  that  a  large  percent  of  these  hospitals  lack  a 
number  of  good  management  practices  which  represented  key  ways  of  Improving 
hospital  operating  efficiency.  The  GAO  study  suggests  that  many  hospitals 
operate  with  distinct  inefficiencies  and  that  identifying  and  reducing  these 
inefficiencies  would  be  desirable  because  of  the  cost  savings  that  would 
result.  This  result  is  quite  believable.  The  absense  incentives  to  maximize 
efficiency  gives  a  hospital  manager  considerable  latitude  to  emphasize  concern 
for  effectiveness  of  care  to  the  detriment  of  efficiency. 

A  methodology  that  would  locate  inefficient  hospitals,  indicate  the 
magnitude  of  the  inefficiency  and  the  general  location  of  that  inefficiency 
could  be  used  to  help  hospital  managers  analyze  and  Implement  programs  to 
improve  efficiency.  In  addition,  it  might  be  possible  to  study  the  management 
techniques  found  in  the  more  efficient  hospitals  and  transfer  them  to  the  less 
efficient  hospitals  as  a  means  of  improving  their  efficiency.  Regulators 
could  use  this  information  to  encourage  and  even  subsidize  the  less  efficient 
hospitals  to  obtain  appropriate  consulting  services  to  help  reduce  their 
inefficiencies.  A  stronger  use  of  these  insights  would  be  to  actually 
penalize  the  less  efficient  hospitals  by  reducing  their  reimbursement  rates  to 

-3- 


levels  of  more  efficient  hospitals  or  by  limiting  their  capital  expansion  as  a 
way  of  applying  direct  pressure  to  improve  efficiency. 

Data  Envelopment  Analysis  is  a  technique  recently  developed  by  A.  Charnes, 
W.  W.  Cooper,  and  E.  Rhodes  ([5],  [6],  and  [7])  which  compares  organizations 
that  use  multiple  inputs  (such  as  labor,  capital,  materials)  to  produce 
multiple  outputs  (such  as  various  types  of  patient  care,  teaching,  research). 
It  compares  these  organizations  and  identifies  those  which  are  relatively 
Inefficient  as  well  as  the  magnitude  of  these  inefficiencies.   This  study 
represents  the  first  field  application  and  test  of  DEA  in  hospital 
evaluations.  The  motivation  for  use  of  DEA  in  the  health  sector  is  twofold. 
1)  The  multiple  outputs  of  hospital  and  specifically  the  case  mix  can  be 
directly  accommodated  with  DEA  and  2)  the  alternative  techniques  are  less 
reliable  and  definitive  In  their  ability  to  identify  hospital  inefficiencies. 

3.  Efficiency  measurement  techniques  for  hospitals 

There  are  two  widely  used  approaches  to  evaluate  hospital  efficiency: 
1.  Ratio  Analysis  (i.e.,  "rules  of  thumb")  -  use  of  various  ratios  for  a 
group  of  comparable  hospitals  to  locate  relationships  which  are 
abnormally  high  or  low  such  as  cost  per  patient  day,  cost  per  patient, 
personnel  full-time  equivalents  per  patient.  Examples  of  this  type  of 
ratio  data  are  Monitrend  reports  of  the  American  Hospital  Association 
and  the  Massachusetts  Rate  Setting  Commission  ratios  used  for  cost 
audits  [11]. 


^DEA  has  been  shown  to  be  theoretically  sound  [5],  [6],  and  [7]  and  has 
been  found  to  be  reliable  in  controlled  applications  [13]  and  has  also  been 
used  in  areas  such  as  education  and  court  systems  efficiency  evaluations. 

-4- 


2.  Econometric  regression  techniques  -  used  to  estimate  hospital  cost 

relationships  and  production  relationships.  Examples  of  this  type  of 
study  are  numerous.  Feldstein's  study  of  hospitals  [9]  reflects  many 
of  the  more  traditional  alternatives  using  econometric-regression 
techniques.  These  approaches  attempt  to  estimate  marginal  cost  per 
patient,  the  breakdown  of  fixed  versus  Variable  cost,  existence  of 
economies  of  scale,  and  the  efficient  rates  of  substitution  between 
inputs. 

Both  ratio  analysis  and  regression  methodologies  have  distinct  limitations 
with  respect  to  evaluating  hospital  efficiency.  These  limitations,  may  be 
summarized  as  follows. 
Ratio  analysis  -  Problems  in  evaluating  hospital  efficiency 

Ratio  analysis  calculates  and  attempts  to  understand  relationship  between 
two  variables  sxich  as  cost  per  day,  cost  per  patient,  etc.  By  their  nature, 
each  ratio  is  limited  to  one  output  and  cannot  easily  accommodate  multiple 
outputs  and  inputs.  If  a  hospital  treated  only  two  types  of  patients  and 
trained  residents,  it  would  have  three  outputs.  Assume,  for  example,  that 
cost  per  patient  day  is  calculated  as  total  number  of  patient  days  t  total 
costs.  This  ratio  would  be  biased  by  the  lack  of  recognition  that  there  are 
two  different  types  of  patients  being  treated  and  because  training  outputs  are 
ignored.  Hence,  a  higher  cost  per  patient  day  could  be  due  to  the  case  mix, 
the  intensity  of  training  activity,  excessive  prices  paid  for  resources  used, 
or  excessive  amounts  of  resources  used  (generally  referred  to  as  operating  or 
technical  inefficiency). 

The  efficient  cost  is  not  known  for  most  hospital  sevlces.  Consequently, 
when  comparing  hospitals,  those  hospitals  with  cost  per  patient  or  cost  per 
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day  some  distance  above  the  mean  cost  might  be  considered  potentially 
inefficient.  There  currently  is  no  way,  however,  to  determine  objectively  how 
far  above  the  mean  is  inefficient  or  even  if  the  mean  is  efficient.  It  is 
conceivable  that  all  the  hospitals  operating  at  the  mean  cost  per  patient  day 
are  also  inefficient.  To  compensate  for  the  unidimension  of  a  single  ratio, 
large  sets  of  ratios  are  developed  as  in  Monitrend  reports.  One  hospital  may 
appear  relatively  efficient  on  one  group  of  ratios  and  inefficient  on  another 
while  another  hospital  may  have  the  opposite  result  for  the  same  ratios. 
There  is  no  objective  means  of  assigning  relative  weights  to  these  ratios. 
Consequently,  it  is  difficult  to  conclude  which  hospitals  are  inefficient 
using  ratio  analysis.  Ratio  analysis  may  be  very  useful  in  identifying  what 
aspect  of  a  hospital's  operation  is  out  of  line  with  the  norm,  but  it  is  of 
limited  help  in  locating  the  inefficient  hospitals  among  a  large  group  of 
hospitals. 
Econometric-Regression  Analysis  -  Problems  in  evaluating  hospital  efficiency 

Regression  analyses  is  more  comprehensive  than  ratios  because  it  can 
accommodate  multiple  ouptputs  and  Inputs,  but  other  significant  problems  are 
encountered.  The  use  of  least  square  regression  techniques  result  in 
estimates  of  average  (or  central  tendency)  relationships  which  are  not 
necessarily  efficient  relationships  as  noted  above. 

A  second  problem  is  an  estimate  of  the  hospital  cost  function  using  this 
technique  results  in  a  mean  relationship  which  does  not  directly  locate 
inefficient  hospitals.  Designation  of  the  relatively  Inefficient  hospitals 
again  requires  that  hospitals  with  costs  some  arbitrary  distance  from  the  mean 
be  labelled  as  potentially  inefficient.  More  importantly,  numerous 
econometric-regression  types  of  hospital  studies  have  been  used  to  identify 
economies  of  scale,  marginal  costs  of  patient  care  and  rates  of  substitution 
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among  outputs  and  Inputs.  These  results  say  nothing,  however,  about  what  the 
efficient  rates  of  substition,  efficient  scale  size,  and  efficient  rates  of 
transformation  are  because  they  are  based  on  an  average  of  the  behavior  of 
efficient  and  Inefficient  hospitals  combined.  Use  of  regression  techniques 
would  only  provide  Insights  into  efficient  hospital  behavior  if  all  the 
hospitals  in  the  study  were  known  to  be  efficient.  While  this  problem  was 
noted  early  on  by  Feldstein  [9],  econometric-regression  techniques  are  among 
the  most  accessible  and  have  consequently  been  widely  used  in  hospital 
Industry  cost  studies. 
Data  Envelopment  Analysis 

DEA  addresses  the  limitations  associated  with  ratio  analysis  and 
regression  techniques  It  is  explicitly  able  to  consider  the  multiple  outputs 
and  inputs  of  a  hospital.  Specifically,  the  multiple  outputs  reflected  in  the 
case  mix  and  the  multiple  resources  used  to  produce  these  services  to  gain  an 
overall  evaluation  of  hospital  efficiency.  In  addition,  it  can  incorporate 
other  hospital  outputs  like  teaching,  research,  and  community  education 
programs  to  gain  a  comprehensive  efficiency  measure  of  hospital  performance. 

Data  Envlopment  Analysis  is  a  linear  programming  technique  which  compares 
a  set  of  an  organization's  actual  inputs  used  to  produce  their  actual  output 
levels.  DEA  locates  those  units  that  are  the  relatively  more  and  less 
efficient  and  measures  the  inefficiency  compared  with  the  more  efficient  units 
in  the  set.  Inefficient  units  are  those  with  an  efficiency  ratio  of  less  than 
1  (E  <  1)  and  those  units  are  strictly  inefficient  compared  to  other  units 
in  the  set.  Units  with  an  efficiency  ratio  of  1  (E  =  1)  are  not  necessarily 
absolutely  inefficient  but  rather  represent  the  "best  practice"  group  of  units 
which  means  that  they  are  not  clearly  inefficient  compared  to  other  units  in 
the  set.  This  situation  arises  because  the  Identity  of  absolutely  efficient 
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hospitals  Is  not  known  because  of  lack  of  knowledge  of  the  truly  efficient 
input-output  relationships.  Hence  a  hospital  that  is  found  to  be  relatively 
efficient  may  also  be  able  to  Improve  its  operating  efficiency.  A  hospital 
that  Is  found  to  be  inefficient  will  have  true  inefficiencies  at  least  as 
large  as  the  amount  located  with  DEA.  An  Inefficient  hospital,  as  identified 
by  DEA,  is  defined  to  have  the  ability  to  produce  its  same  level  of  outputs 
(patient  care,  teaching)  with  fewer  Inputs  based  on  the  actual  output-input 
levels  of  hospitals  that  were  compared  with  the  inefficient  hospital. 

The  DEA  model  is  described  in  the  appendix  to  this  paper.   (See  exhibit  3.) 
To  apply  DEA,  it  is  necessary  to  identify  and  obtain  the  data  for  set  of 
outputs  and  inputs  relevant  to  the  hospitals'  operations.  Each  of  the  outputs 
and  inputs  need  only  be  measured  in  their  natural  physical  units  without  the 
need  to  use  a  homogeneous  measurement  unit  like  dollars.  For  example,  DEA  can 
Include  as  outputs  the  number  of  each  patient  diagnosis  type  treated  and  the 
number  of  each  type  of  individual  trained,  and  it  can  Include  inputs  measured 
In  units  of  full-time  equivalents  of  each  personnel  type,  number  of  beds 
available,  etc. 
4,  Field  Test  of  DEA  to  evaluate  Teaching  Hospital  Efficiency 

A  set  of  teaching  hospitals  in  Massachusetts  were  used  to  evaluate  the 
ability  of  DEA  to  locate  relatively  inefficient  hospitals.  This  researcher 
had  no  a  priori  knowledge  about  which  of  these  hospitals  were  relatively 
Inefficient  nor  was  any  accepted  benchmark  available  that  independently 
Indicated  which  of  these  hospitals  were  more  or  less  efficient.  In  lieu  of  an 
absolute  benchmark  of  efficiency,  a  panel  of  hospital  experts  including 
regulators,  managers,  and  hospital  management  consultants  that  were  familiar 
with  the  hospitals  in  the  state  were  enlisted  to  evaluate  the  accuracy  of  the 
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DEA  results  and  to  help  identify  the  outputs  and  inputs  that  were  relevant  for 
the  efficiency  evaluation.   (The  procedures  followed  are  described  in  greater 
detail  in  [14].) 
Choice  of  Sample 

The  state  rate  setting  commission  had  already  adoped  a  ratio  analysis 
approach  to  evaluate  hospitals  whereby  a  hospital  over  one  standard  deviation 
above  the  mean  cost  per  day  or  mean  cost  per  patient  was  deemed  potentially 
inefficient  [11].  This  ratio  analysis  was  applied  within  groups  of  hospitals 
that  were  designated  as  "comparable"  groups.  These  comparable  groups  were 
developed  by  the  rate  setting  commission  using  cluster  analysis  modified  by 
discussions  with  the  hospitals.  One  of  these  comparable  groups  of  hospitals, 
a  set  of  teaching  hospitals,  was  selected  for  this  study  and  the  data  used  was 
from  reports  required  to  be  submitted  annually  to  the  rate  setting  commission 
by  each  hospital  in  the  state  [12].  Data  was  available  for  seven  of  the  nine 
hospitals  in  this  group,  so  the  DEA  was  used  to  evaluate  just  those  seven 
hospitals.  The  study  focused  on  one  part  of  the  hospital,  the  medical 
surgical  area  as  defined  in  [12],  to  reduce  the  complexity  of  analyzing  the 
efficiency  in  an  entire  hospital  and  because  this  represented  the  largest 
single  cost  area  separately  reported  on  by  hospitals  in  the  state.  Data  from 
1976  was  used  because  it  was  believed  that  some  of  the  negative  connotations 
of  being  labelled  "inefficient"  would  be  diffused  if  the  study  tested  DEA 
using  a  prior  year.  The  study  began  in  1979  so  that  the  1976  year  of 
operations  was  not  so  old  as  to  have  faded  from  the  memory  of  those  familiar 
with  these  hospitals. 
Identification  of  relevant  outputs  and  inputs 

The  identification  of  the  relevant  outputs  and  inputs  had  to  be  based  on 
an  understanding  of  what  resources  are  used  to  provide  the  types  of  services 


-9- 


offered  In  the  Medical  Surgical  (MS)  area.  In  this  study,  the  variable 
selection  was  also  tempered  by  the  data  that  would  be  available  from  public 
reports  from  these  hospitals. 

The  outputs  and  inputs  used  are  described  in  table  1. 

The  elaborate  process  of  defining  these  outputs  and  inputs  is  described  in 
exhibit  1  in  the  appendix.  The  process  began  with  a  list  of  all  the 
identifiable  and  relevant  direct  outputs  and  inputs  of  the  medical  surgical 
area.  This  list  was  refined  by  eliminating  input  measures  like  square  feet  of 
building  space  which  was  believed  to  be  less  directly  associated  with 
efficiency  and  which  was  already  reflected  in  part  by  one  of  the  input 
measures  used  -  bed  days  available.  The  most  problematic  compromise  was  the 
use  of  only  two  case  types,  over  and  under  65  year  of  age  rather  than  the  more 
direct  and  complete  measures  like  diagnostic  related  groups  (DRG's)  [3]. 

While  age  does  appear  to  be  a  key  factor  in  the  resource  required  to  treat 
a  patient  [3]  [13],  age  alone  is  an  incomplete  measure.  Proceeding  with  only 
age  as  a  case  mix  breakdown  would  mean  that  the  DEA  results  might  be  skewed 
because  other  case  mix  dimensions  of  the  hospitals'  outputs  are  not  accounted 
for.  This  problem  is  somewhat  reduced  in  the  set  of  hospitals  used  because 
they  were  already  believed  to  be  "comparable",  i.e.,  all  of  these  hospitals 
had  a  relatively  severe  (resource  intensive)  set  of  patient  diagnoses 
characteristic  of  these  teaching  hospitals.  In  addition,  the  Medical  Surgical 
area  was  moderated  in  its  case  mix  complexity  because  patients  were  generally 
admitted  to  this  area  only  after  being  stabilized  in  the  intensive  care, 
emergency  care,  or  operating  recovery  room  areas.  DEA  would  evaluate  these 
hospitals'  use  of  the  three  inputs  in  table  1  to  produce  patient  care  and 
training  as  measured  by  the  four  outputs  in  table  1.  To  the  extent  that  there 
are  other  outputs  and  Inputs  not  included,  the  DEA  results  may  be  less  than 
comprehensive.  The  experts  agreed  that  the  data  that  was  specified  in  table  1 
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Table  1 


The  Inputs  and  outputs  selected  and  used  for  the  DEA  analysis  were  as  follows: 


(1)  FTE's 

(2)  Supply  $'s 

(3)  Bed  days 


INPUTS 

Full-time  Non-Physician  Equivalents  specifically 
employed  in  the  Medical  Surgical  (MS)  area 
during  Fiscal  Year  1976  (FY) 

Total  dollar  value  of  supplies  and  purchased 
services  used  in  MS  area  during  FY  1976 

Number  of  bed  days  available  in  MS  area  during 
FY  1976 


(1)  Patient  days 
with  age  ^  65 

(2)  Patient  days 
age  <  65 

(3)  Nurses  trained 


(4)  Interns/ 
Residents 
trained 


OUTPUTS 

-  Number  of  patient  days  of  care  in  MS  area  for 
patients  65  or  over  during  FY  1976 

-  Number  of  days  of  care  for  patients  under  65 
during  FY  1976 

-  Number  of  nursing  students  trained  during  FY 
1976  in  their  first,  second  or  third  years  of 
the  hospitals'  nursing  schools 

-  Number  of  interns  and  residents  receiving  one 
year  of  training  at  the  hospital  during  FY  1976 
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was  a  reasonable  list  of  relevant  outputs  and  Inputs  that  characterize 
medical-surgical  area  activities  but  that  a  more  detailed  case  mix  measure 
would  have  been  a  desirable  addition  if  it  were  available. 

The  actual  output  and  input  data  used  are  reported  in  exhibit  2  of  the 
appendix. 
PEA  results  versus  the  use  of  ratio  analysis 

The  DEA  results  are  summarized  in  Table  2  which  indicates  that  hospital  D 
and  G  are  relatively  inefficient  compared  with  the  other  hospitals  in  the  data 
set,  i.e.  efficiency  rating  of  less  than  1.0.   (The  hospital  names  are 
disguised  but  these  names  were  made  available  to  the  expert  panel.) 

The  rate  setting  commission's  ratio  analysis  suggested  that  only  hospital 
C  (and  specifically  not  hospitals  D  and  G)  may  be  relatively  inefficient  in 
the  prices  they  pay  for  inputs  and/or  the  amount  of  inputs  used  (Table  2, 
columns  4  and  5).  DEA  goes  beyond  these  ratios  and  suggests  that  hospitals  D 
and  G  are  also  relatively  inefficient  and  that  compared  to  other  hospitals  in 
this  group,  these  inefficient  hospitals  should  be  able  to  produce  their  same 
level  of  services  (outputs)  with  fewer  inputs  and,  therefore,  at  lower  cost. 
Interpretation  of  DEA  results 

The  meaning  of  the  inefficient  rating  derived  from  DEA  can  be  understood 
by  examining  the  results  for  hospital  D.  DEA  indicates  that  hospital  D  is 
inefficient  with  an  efficiency  rating  of  .88  based  on  comparison  of  hospital  D 
with  all  seven  hospitals.  More  specifically,  DEA  indicates  that  the 
Inefficiency  was  located  and  measured  by  comparing  hospital  D  with  its 
efficiency  reference  set  hospitals  A,  C,  and  E,  noted  in  table  2.  This 
Information  is  a  direct  output  of  DEA.  By  identifying  the  efficiency 
reference  set,  DEA  allows  one  to  focus  on  a  subset  of  these  hospitals  to 
better  understand  the  inefficiencies  present.  This  comparison  is  illustrated 
in  table  3  which  indicated  that  a  weighted  composite  of  the  efficiency 
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reference  set  hospitals  would  yield  a  hypothetical  hospital  that  produces  as 
much  or  more  outputs  as  the  Inefficient  hospital  D  but  also  uses  less  inputs 
than  D.  In  this  example,  the  composite  is  constructed  by  applying  the 
weights,  (the  dual  variables  from  the  DEA-linear  program),  of  0.138,  0.296, 
and  0.498,  respectively,  to  the  actual  outputs  and  inputs  of  hospitals  A,  C, 
and  E.  Columns  (4),  (5),  and  (6)  of  table  3  indicate  that  a  combination  of 
the  actual  operations  of  these  three  hospitals  would  result  in  a  hypothetical 
hospital  that  would  use  55  fewer  FTE's,  $182,330  fewer  supply  dollars,  and 
9090  fewer  bed  days  to  produce  the  same  amount  of  patient  care  and  34 
additional  units  of  training. 
Experts  assessment  of  PEA  results 

The  experts  agreed  that  the  two  hospitals  identified  as  inefficient,  D  and 
G,  was  a  reasonable  and  believable  result  though  one  expert  expressed  some 
doubt  about  the  magnitude  of  the  inefficiency  found  in  hospital  D. 

The  reaction  of  the  experts  provides  added  insights  into  the  need  for 
techniques  like  DEA.  These  experts  were  not  able  to  provide  a  ranking  of 
relative  efficiency  before  the  DEA  results  were  obtained  for  two  reasons:   (a) 
they  do  not  generally  evaluate  hospital  performance  based  on  output-input 
efficiency  criteria  but  rather  were  more  aware  of  cost  per  patient  and  cost 
per  day  types  of  data,  and  (b)  the  experts  were  not  as  knowledgeable  about  an 
individual  department  like  Medical  Surgical  but  rather  had  general  impressions 
of  these  hospitals'  overall  performance.  Their  reaction  to  the  validity  of 
the  DEA  findings  were,  nevertheless,  very  strong  and  they  indicated  that  the 
two  hospitals  identified  as  inefficient  reflect  a  believable  result  based  on 
knowledge  of  these  hospitals  and  the  quality  of  their  management.  DEA 
effectively  located  two  inefficient  hospitals  that  would  not  be  so  identified 
or  measurable  with  commonly  used  techniques. 
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Table  2 


Comparison  of  teaching  Hospitals'  Medical  Surgical  (MS)  area 


Hospital** 
(1) 

DEA 
Efficiency 
Rating 
(2) 

Efficiency 
Reference 
Set  (ERS) 
(3) 

MS  Cost 

Per  Patient 

Day 

(A) 

MS  Cost 

Per  Patient 

(5) 

A 

1.0 

- 

$34 

$408 

B 

1.0 

- 

38 

418 

C 

1.0 

- 

39 

429 

D 

0.88 

A,  C,  E 

32 

407 

£ 

1.0 

- 

27 

243 

F 

1.0 

- 

29 

348 

G 

0.93 

E 

36 

324 

Average  Cost 

$34.29 

$368.14 

Standard  Deviation 

$  4.27 

$  62.45 
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Evaluation  of  PEA  results  by  management  of  inefficient  hospital  D 

To  further  test  the  validity  of  these  results,  they  were  reviewed  with  the 
director  and  financial  officer  of  inefficient  hospital  D.  They  agreed  that 
the  medical  surgical  areas  of  these  seven  hospitals  were  comparable  and  that 
the  outputs  and  Inputs  selected  reasonably  captured  the  key  Inputs  and  outputs 
of  the  area.  While  the  case  mix  data  was  questioned,  they  did  not  feel  that 
their  case  mix  was  more  severe  or  resource  consuming  than  others  in  the  group 
so  that  this  was  not  believed  to  be  the  cause  of  the  Inefficiency  identified 
in  hospital  D. 

They  located  three  potential  explanations  for  the  inefficiencies 
identified  by  DEA. 

-  The  supply  cost  data  was  found  to  be  overstated  by  $141,000  due  to  an 
accounting  tranfer  that  appears  elsewhere  in  the  report  to  the  rate 
setting  commission  and  which  was  subsequently  determined  to  be  a 
peculiarity  that  affected  hospital  D's  report  but  did  not  affect  the  data 
for  the  other  six  hospitals. 

-  The  bed  days  was  determined  by  management  to  be  excessive  and  were 
reduced  subsequent  to  1976  by  6933  bed  days  as  was  verified  by  examining 
subsequent  years'  agency  reports. 

-  Personnel  levels  of  hospital  D  were  compared  with  other  hospitals  in  the 
group  and  noted  that  they  were  somewhat  higher  than  comparable  hospitals 
by  about  5.4  FTE's,  but  that  this  was  a  result  of  a  conscious  decision  to 
maintain  a  somewhat  larger  staff  to  provide  more  personalized  patient  care. 

In  summary,  management  of  hospital  D  found  a  data  problem,  an  excessive 
level  of  bed  days  available  which  was  subsequently  adjusted  and  an  excess  FTE 
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level  that  continued  to  exist. 

The  DEA  evaluation  was  rerun  with  the  corrected  supply  dollar  level  and 
with  the  reduced  level  of  beds  to  determine  if  these  adjustments  would  make 
hospital  D  efficient  compared  with  the  other  six  hospitals.  It  was  found  that 
hospital  D  was  still  Inefficient  compared  to  the  other  six  hospitals,  but  with 
a  higher  efficiency  rating  of  .96  instead  of  the  original  rating  of  .88. 
Hence,  hospital  D  remained  inefficient  and  this  was  believed  to  be  due  in  part 
to  personnel  levels.  Another  DEA  evaluation  which  reduced  FTE's  by  the  amount 
calculated  by  management  (5. A  units)  and  also  reflected  the  above  adjustments 
were  found  to  be  adequate  to  make  hospital  D  efficient. 

Hospital  D  could  become  relatively  efficient  by  making  adjustments  to  its 
inputs  of  a  lower  magnitude  than  was  indicated  by  DEA  as  indicated  in  table 
4.  This  reflects  the  existence  of  a  number  of  alternative  paths  that  any 
Inefficient  hospital  may  select  to  become  relatively  efficient.  A  number  of 
alternative  paths  Including  the  one  noted  in  column  1,  table  4  is  available 
directly  from  DEA.  Other  paths  that  are  considered  more  practical  by 
management  may  also  make  a  hospital  relatively  efficient.  The  ability  of 
these  other  alternative  paths  to  make  an  inefficient  hospital  become 
relatively  efficient  can  also  be  evaluated  using  DEA  for  sensitivity  analysis 
as  was  described  above.  In  the  case  of  hospital  D,  the  adjustments  that 
management  considered  necessary  to  make  it  efficient  as  represented  in  column 
2  of  table  4  was  found  to  be  one  adequate  path  to  relative  efficiency. 


-17- 


Table  4 


(1) 
Excessive  Inputs 
based  on  DEA  evaluation 
of  Hospital  D  (Table  3) 


(2) 
Input  Adjustments 
and  Excesses  Noted 
by  Hospital  D  Management 


(3) 

Cause  of  Excess 
Inputs 


FTE's 


55 


5.4 


Intentionally  richer 
staffing 


Bed  days 
Available 


9090 


Supply  $     $182,230 


6935 


141,000 


Reduction  of  19  beds 
to  compensate  for  low 
occupancy  rate 

Transfer  due  to  unique 
accounting  system 


-18- 


Conclusion  of  the  field  application  of  PEA  to  hospitals 

The  key  results  of  this  field  study  may  be  summarized  as  follows : 

-  DEA  accurately  located  two  inefficient  hospitals  that  would  not  be 
identifiable  using  the  types  of  ratio  analysis  used  by  the  state  rate 
setting  commission. 

-  DEA  could  explicitly  consider  different  outputs  measured  in  their 
natural  units  and  was  able  to  identify  as  well  as  measure  the  relative 
magnitude  of  inefficiency  present,  i.e.,  the  amount  of  potential  resource 
reduction  possible  if  these  inefficient  hospitals  attained  the  operating 
efficiency  of  the  relatively  more  efficient  hospitals  in  the  study. 

-  DEA  results  could  be  understood,  interpreted,  and  accepted  by 
administrators  familiar  with  these  hospitals  to  help  locate  the  source  of 
the  inefficiency.  This  could  lead  to  managerial  action  to  reduce  these 
inefficiencies  and/or  to  clarify  the  magnitude  of  the  inefficiencies  or 
slack  that  management  chose  to  incorporate  into  their  operating  plans. 

5.   Future  Use  of  DEA  for  Hospital  Efficiency  Evaluations 

The  field  test  result  of  this  study  buttressed  by  the  theoretical 
formulation  and  other  tests  of  DEA  described  in  [5],  [7]  and  [13]  suggest  that 
DEA  is  a  promising  tool  to  evaluate  hospital  efficiency.  These  studies 
indicate  that  DEA  is  reliable  in  the  location  of  relatively  inefficient  units 
and  in  its  ability  to  suggest  the  general  magnitude  of  input  reductions 
required  to  make  an  inefficient  unit  relatively  efficient.  It  is  particularly 
useful  in  hospital  applications  because  it  can  simultaneously  accommodate 
multiple  outputs  and  inputs  and  does  not  require  specific  knowledge  of  the 
efficient  amount  of  inputs  required  for  each  hospital  output. 
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Operational  Issues  in  applying  PEA  to  hospitals 

Input-Output  specifications  —  a  key  Ingredient  In  the  DEA  evaluation  Is 
the  location  and  measurement  of  the  relevant  Inputs  and  outputs.  The 
advantage  of  DEA  over  other  techniques  Is  that  each  Input  and  output  can  be 
measured  in  their  natural  physical  units  without  the  need  to  apply  a  weighting 
system  to  collapse  these  different  units  into  dollars  or  any  other  single  unit 
measure.  Hence,  it  is  sufficient  to  know,  for  example,  how  many  patients  of 
each  diagnosis  was  treated  as  an  output  measure  and  how  many  FTE's  of  each 
personnel  type  are  utilized  to  produce  the  outputs.  This  is  of  major 
significance  because  it  means  that  case  mix  can  be  explicitly  considered  in 
the  efficiency  evaluation  by  considering  each  case  type  (e.g.,  diagnostic 
related  group  case  type)  to  be  a  separate  output. 

Defining  and  measuring  the  relevant  physical  Inputs  and  outputs  may 
require  additional  effort,  since  many  hospital  Information  systems  tend  to 
emphasize  and  collect  only  the  dollar  cost  of  Inputs  due  to  the  Importance  of 
the  reimbursement  systems  and  the  widespread  concern  about  the  dollar  cost  of 
health  care.  Similarly,  comprehensive  case  mix  output  data  are  only  beginning 
to  be  captured  in  a  form  which  is  useable  for  assessment  of  hospital 
efficiency  via  DEA  or  other  techniques.  These  data  collection  efforts  are 
needed  to  help  Identify  a  path  to  reducing  inefficiency  and  costs,  regardless 
of  the  methodology  to  be  used.  Specifically,  a  hospital  located  as 
inefficient  using  this  data  with  DEA  will  be  strictly  inefficient  and 
consequently  will  have  the  ability  to  produce  the  same  level  of  outputs  with  a 
reduced  level  of  Inputs  which  can  translate  Into  lower  costs. 
Use  of  DEA  to  hospitals  to  improve  technical  efficiency 

Based  on  this  pilot  study,  DEA  may  be  applied  to  hospitals  in  the 
following  ways. 

-  Segments  of  hospitals  like  the  Medical  Surgical  area  as  well  as  an 

-20- 


entire  hospitals'  operation  may  be  compared  to  locate  the  relatively 
Inefficient  hospitals. 

-  Location  of  relatively  inefficient  hospitals  can  be  used  to  allocate 
resources  to  reduce  these  inefficiencies,  i.e.,  remedial  resources  can  be 
focused  on  the  hospitals  most  likely  to  have  inefficiencies  that  can  be 
reduced  by  in  depth  study  of  their  operations.  This  may  provide  a  basis 
for  regulators  or  insurers  to  encourage  and  perhaps  subsidize  the  hiring 
of  consultants  or  auditors  to  evaluate  ways  of  increasing  the  inefficient 
hospitals'  efficiency  levels  which  should  ultimately  translate  into  lower 
operating  costs  or  increased  service  levels. 

-  DEA  results  can  be  used  by  managers  or  regulators  to  compare  techniques 
used  in  relatively  more  and  less  efficient  hospitals  and  to  identify 
techniques  used  by  more  efficient  hospitals  that  can  be  transferred  to  and 
adopted  by  the  inefficient  hospitals.  As  suggested  in  the  field  study, 
DEA  not  only  locates  the  Inefficient  units  but  it  also  locates  the 
relatively  efficient  hospitals  against  which  that  hospital  was  most 
directly  found  to  be  Inefficient.  In  this  way,  the  group  of  hospitals  to 
be  compared  is  significantly  reduced  from  the  original  size  of  the  groups 
being  evaluated.  In  this  study,  hospital  D  needed  only  be  compared  to 
three  hospitals  (A,  C,  and  E)  rather  than  all  of  the  six  hospitals  and 
hospital  G  needed  only  be  compared  with  hospital  E. 

-  Regulatory  organizations  can  use  DEA  to  select  out  the  more  efficient 
hospitals  as  a  basis  for  rate  setting.  They  could  require  that  less 
efficient  hospitals  receive  no  more  than  the  rates  required  to  reimburse 
the  more  efficient  hospitals  for  similar  services.  This  might  provide  an 
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incentive  for  less  efficient  hospitals  to  become  at  least  as  efficient  as 
the  relatively  more  efficient  hospitals  in  the  data  set. 

-  Use  of  DEA  can  encourage  managers  to  specifically  consider  physical 
input-output  efficiency  as  distinct  from  other  performance  dimensions  such 
as  maximization  of  reimbursement  revenues  and  minimizing  of  material  costs 
and  salary  levels  as  a  direct  means  of  reducing  health  care  costs.  This 
suggests  that  DEA  has  capabilities  not  found  in  other  measurement 
techniques  such  as  ratio  analysis.  At  the  same  time,  other  techniques  can 
address  efficiency  dimensions  not  addressed  by  DEA  such  as  the  cost  per 
FTE,  cost  per  unit  of  a  drug.  DEA  is,  therefore,  a  useful  complement  to 
other  techniques  rather  than  a  replacement  for  other  techniques. 
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Appendix  —  Exhibit  3 
The  DEA  Model 

DEA  measures  the  efficiency  of  hospital  o  compared  with  the  n  hospital  in 
the  data  set  as  follows: 


Ob.jectlve; 


s 

Z 

mcix  E  =  r=l 
o   

m 


u^y 


ro 


L   v.x. 
i  io 


where  o  is  the  hospital  being  evaluated  in  the  set 
of  j  =  1,  .  .  .,  n  hospitals.   (This  analysis  is 
run  repetitively  with  each  hospital  in  the  objective 
function  to  derive  an  efficiency  rating  for  each  of 
the  n  hospitals) . 


1=1 


Constraints: 


Less  than 

Unity 

Constraints 


Positlvity 
Constraints 


s 

■1  >       r=l  "^  ^-^ 
m 

Z  V  X 
1=1  ^  ^J 


0  <  ur  ;  r  =  1, 
0  <  Vi  ;  1  =  1, 


J  =  1, 


(1) 


.,  s 

• ,  m 


Data: 

Outputs: 

Inputs: 


yj.4  =  observed  amount  of  r^"  output  for  the  j^"  hospital 
xjj  =  observed  iimount  of  1*^"  input  for  the  j^"  hospital 

The  data  used  for  each  hospital  are  the  y  ,  outputs;  and  the  x. .  Inputs. 

The  u  ,  V.  values  are  determined  from  the  data  by  the  above  model.  DEA 

provides  an  ex  post  evaluation  of  how  efficient  each  hospital  was  with  the 

actual  inputs  (x.  .)  used  to  produce  its  outputs  (y  .)  without  explicit 

knowledge  of  the  input-output  relationships  it  used.  The  weights  in  the 

form  of  the  u  and  the  v  are  not  known  or  given  a  priori.  They  are, 

instead,  calculated  as  (u  ,  v  )  values  to  be  assigned  to  each  input  and 

output  in  order  to  maximize  the  efficiency  rating  — E  —  of  the 

hospital  being  evaluated.  That  is,  the  solution  sought  is  the  set  of 

(u  ,  v.)  values  that  will  give  the  hospital  being  rated  the  highest 
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efficiency  ratio,  E  ,  but  not  result  in  an  input-output  ratio  exceeding 

1  (100%  efficiency)  when  applied  to  any  and  all  other  hospitals  in  the  data 

set.   (See  [5]  and  [7]  for  further  details). 

Applying  DEA  to  a  set  of  hospitals'  results  in  an  efficiency  rating  for 
each  hospital  of  1  (relatively  efficient)  or  less  than  1  (relatively 
inefficient).  These  ratings,  however,  represent  relative  efficiencies  based 
on  comparison  of  hospitals  in  the  data  set  (j  =  l...n  hospitals).  A  hospital 

that  is  found  to  be  inefficient  (h  <  1)  is  strictly  inefficient 

o  ^ 

compared  to  other  hospitals  in  the  data  set  as  is  shown  in  [13]. 

Note  that  the  u  ,  v,  values  calculated  by  DEA  are  objectively 
determined  weights  which  may  not  correspond  to  relative  values  that  a 
hospital  would  assign  to  outputs  and  inputs.  This  is  actually  a  strength 
and  not  a  weakness  of  DEA.  A  hospital  located  as  efficient  using  DEA  Is  so 
identified  only  after  all  possible  weights  have  been  considered  to  give  that 
hospital  the  highest  rating  possible  consistent  with  the  contraint  that  no 
hospital  in  the  data  set  can  be  more  than  100%  efficient.  Hence,  any  other 
set  of  weights  applied  to  all  hospitals  would  only  make  an  inefficient 
hospital  appear  less  efficient,  i.e.,  DEA  gives  the  benefit  of  the  doubt  to 
each  hospital  in  calculating  the  efficiency  value. 

The  DEA  evaluation  also  provides  insights  far  beyond  the  identification 
of  the  inefficient  hospitals  (see  [13])  as  is  illustrated  in  the  hospital 
application  in  table  3  and  further  described  in  [13]. 
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